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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address ~ 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

• If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1)D Responsive to communication(s) filed on . 

2a)D This action is FINAL. 2b)^ This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-20 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) \ is/are allowed. 

6) ^] Claim(s) 1-7 and 9-20 is/are rejected. 

7) ^ Claim(s) 8 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 1 1 9 

1 2) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 1 9(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2.D Certified copies of the priority documents have been received in Application No. . 



3.D Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 

Claim Rejections - 35 USC § 103 

1 . The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as 
set forth in section 102 of this title, if the differences between the subject matter sought to be 
patented and the prior art are such that the subject matter as a whole would have been obvious 
at the time the invention was made to a person having ordinary skill in the art to which said 
subject matter pertains. Patentability shall not be negatived by the manner in which the invention 
was made. 

2. Claims 1-7 and 9-15 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Chen et al (6,677,982) in view of Burt (5,473,364) and Yang (6,700,999 B1). 

Regarding claims 1, 9, 14, and 15, Chen et al discloses a system/method for 
identifying a location of an object in a physical scene comprising: 

means for identifying a first and a second virtual surface being planar in the 
physical scene (Fig. 1 , 704); 

means for analytically constructing an approximate disparity set of the 
first/second virtual surface (Fig. 1, 706); 

a stereo camera including camera parameters acquiring a main and a reference 
image of the scene (Fig. 2); and 

means for warping the reference image according to the first/second disparity set 

(716); 

Chen et al does not particularly disclose means for subtracting each pixel of the 
warped reference image from a corresponding pixel of the main image to determine a 
depth residual of each pixel, and means for identifying each pixel having a non-zero 
depth residual with a surface of the object not being coincident with the virtual surface. 

However, Burt teaches a video technique comprising means for subtracting (Fig. 

3. 310) each pixel of the warped reference image (306) from a corresponding pixel of 
the main image (300-2) to determine a depth residual of each pixel. 
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Furthermore, Yang teaches means (binary segmentation mask) for identifying 
each pixel having a substantial non-zero value with a surface of the object (face) not 
being coincident with the virtual surface (background)(Fig. 1, P160 and Fig. 2). 

Therefore, it would have been obvious to a person of ordinary skill in the art 
employing a system/method for identifying a location of an object in a physical scene as 
taught by Chen et al to incorporate the Burt's teaching so as to subtract each pixel of 
the warped first/second reference images from a corresponding pixel of the main image 
to determine a first and a second depth residual of each pixel and also incorporate the 
Yang's teaching so as to identify each pixel having a non-zero depth residual with a 
surface of the object not being coincident with the virtual surface for an efficient depth 
segmentation in a physical scene, thereby being able to distinguish foregrounds from 
backgrounds. 

Regarding claims 2 and 16, Chen et al teaches the virtual surface (background) 
having an associated margin to form a virtual volume (depth or 3-D) near the virtual 
surface with a thickness equal to the margin (abs.). 

Regarding claims 3-4, 17 and 18, Yang teaches an arbitrary surface 
(background) defined as partially tangible and partially in a space of the physical scene 
(col. 6, lines 27-39). 

Regarding claims 5 and 19, Yang teaches binary segmentation mask (col. 6, 
lines 27-39), and comparing each pixel entry to a predetermined threshold so as to 
determine either the face region (object) or the background. Therefore, it would have 
been obvious to set each depth residual less than a predetermined threshold to zero, 
and set all other depth residuals to one to generate a binary segmentation mask. 

Regarding claims 6 and 20, Burt teaches moving object in a stereo video of the 
scene (Fig. 3), and Yang teaches binary segmentation mask as discussed above. 

Regarding claim 7, the Examiner takes official notice that applying polynomial 
interpolation to generate a disparity (displacement) is conventionally well known in the 
art. Note (6,141,104; abs.). 
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Therefore, it would have been obvious to apply polynomial interpolation to 
generate a disparity by acquiring a set of point correspondence from a calibration pair of 
images. 

Regarding claim 10, Chen et al teaches a touching of the virtual surface by the 
object from the depth disparities (abs.). 

Regarding claim 11, the Examiner takes official notice that a projector is 
conventionally well known in the art. 

Therefore, it would have been obvious to illuminate the scene and the object with 
a dynamic projector so as to enhance the lighting. 

Regarding claim 12, Yang teaches contrast image (P422-6). 

Regarding claim 13, Yang teaches performing segmentation (Fig. 1, P330). 
Chen et al discloses the virtual volume (abs.). 

Therefore, it would have been obvious to perform volume segmentation 
according to virtual volume to locate an object. 

Allowable Subject Matter 

3. Claim 8 is objected to as being dependent upon a rejected base claim 1 , but 
would be allowable: if claim 8 is rewritten in independent form including all of the 
limitations of the base claim 1 and any intervening claims. 

Dependent claim 8 recites the novel features comprising a linear system having 
an equation (see claim 8). 

Accordingly, if the amendments are made to the claims listed above, and if 
rejected claims are canceled, the application would be placed in condition for allowance. 

Conclusion 

4. The prior art made of record and not relied upon is considered pertinent to 
Applicant's disclosure. 

A) Schulz et al (6,141,1 04), System for determination of a location in 3-D 

space. 
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B) Madden et al (6,297,825 B1 ), Temporal smoothing of scene analysis data for 
image sequence generation. 

C) Chiang et al (6,144,701 ), Stereoscopic video coding and decoding apparatus 
and method. 

D) Cahill et al (6,507,665 B1), Method for creating environment map containing 
information extracted from stereo image pairs. 

5. Any inquiry concerning this communication or earlier communications from the 
Examiner should be directed to Shawn S An whose telephone number is 703-305-0099. 
The examiner can normally be reached on Flex hours (10). 

6. The fax phone number for the organization where this application or proceeding 
is assigned is 703-872-9306. 

7. Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 




SSA 

Primary Patent Examiner 
7/29/04 
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